Are COVID-19 systematic reviews up to date and can we tell? A cross-sectional study.

BACKGROUND: COVID-19 led to a rapid acceleration in the number of systematic reviews. Readers need to know how up to date evidence is when selecting reviews to inform decisions. This cross-sectional study aimed to evaluate how easily the currency of COVID-19 systematic reviews published early in the pandemic could be determined and how up to date these reviews were at the time of publication. METHODS: We searched for systematic reviews and meta-analyses relevant to COVID-19 added to PubMed in July 2020 and January 2021, including any that were first published as preprints. We extracted data on the date of search, number of included studies, and date first published online. For the search date, we noted the format of the date and where in the review this was reported. A sample of non-COVID-19 systematic reviews from November 2020 served as a comparator. RESULTS: We identified 246 systematic reviews on COVID-19. In the abstract of these reviews, just over half (57%) reported the search date (day/month/year or month/year) while 43% failed to report any date. When the full text was considered, the search date was missing from 6% of reviews. The median time from last search to publication online was 91 days (IQR 63-130). Time from search to publication was similar for the subset of 15 rapid or living reviews (92 days) but shorter for the 29 reviews published as preprints (37 days). The median number of studies or publications included per review was 23 (IQR 12-40). In the sample of 290 non-COVID SRs, around two-thirds (65%) reported the search date while a third (34%) did not include any date in the abstract. The median time from search to publication online was 253 days (IQR 153-381) and each review included a median of 12 studies (IQR 8-21). CONCLUSIONS: Despite the context of the pandemic and the need to easily ascertain the currency of systematic reviews, reporting of the search date information for COVID-19 reviews was inadequate. Adherence to reporting guidelines would improve the transparency and usefulness of systematic reviews to users.

Conceptualizing the reporting of living systematic reviews.

OBJECTIVES: As part of an effort to develop an extension of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement for living systematic reviews (LSRs), we discuss conceptual issues relevant to the reporting of LSRs and highlight a few challenges. METHODS: Discussion of conceptual issues based on a scoping review of the literature and discussions among authors. RESULTS: We first briefly describe aspects of the LSR production process relevant to reporting. The production cycles differ by whether the literature surveillance identifies new evidence and whether newly identified evidence is judged to be consequential. This impacts the timing, content, and format of LSR versions. Second, we discuss four types of information that are specific to the reporting of LSRs: justification for adopting the living mode, LSR specific methods, changes between LSR versions, and LSR updating status. We also discuss the challenge of conveying changes between versions to the reader. Third, we describe two commonly used reporting formats of LSRs: full and partial reports. Although partial reports are easier to produce and publish, they lead to the scattering of information across different versions. Full reports ensure the completeness of reporting. We discuss the implications for the extension of the PRISMA 2020 statement for LSRs. CONCLUSION: We argue that a dynamic publication platform would facilitate complete and timely reporting of LSRs.

Most published systematic reviews of remdesivir for COVID-19 were redundant and lacked currency.

OBJECTIVE: To investigate the completeness and currency of published systematic reviews of remdesivir for COVID-19 and to compare this with a living guidelines approach. STUDY DESIGN AND SETTING: In this cross-sectional study, we searched Europe PMC on May 20, 2021 for systematic reviews of remdesivir (including preprints, living review updates). Completeness and currency were based on the inclusion of four major randomized trials of remdesivir available at the time of publication of the review (including as preliminary results and preprints). RESULTS: We included 38 reviews (45 reports), equivalent to a new publication every 9 days. 23 (51%) reports were out of date at the time of publication. Eleven reviews that were current on publication had a median survival time of 10 days (range 4-57). A third of reviews cited other systematic reviews, but only four provided justifications for why another review was necessary. Eight (21%) of the reviews were registered in PROSPERO. The Australian COVID-19 Clinical Evidence Taskforce living guidelines were updated within 14 days for three of the remdesivir trials, and within 28 days for the fourth. CONCLUSION: There was considerable duplication of systematic reviews of remdesivir, and half were already out of date at the time of publication.

Protocol: Benefits and harms of remdesivir for COVID-19 in adults: A systematic review with meta-analysis.

BACKGROUND: Effective drug treatments for Covid-19 are needed to decrease morbidity and mortality for the individual and to alleviate pressure on health care systems. Remdesivir showed promising results in early randomised trials but subsequently a large publicly funded trial has shown less favourable results and the evidence is interpreted differently in clinical guidelines. Systematic reviews of remdesivir have been published, but none have systematically searched for unpublished data, including regulatory documents, and assessed the risk of bias due to missing evidence. METHODS: We will conduct a systematic review of randomised trials comparing remdesivir to placebo or standard of care in any setting. We will include trials regardless of the severity of disease and we will include trials examining remdesivir for indications other than Covid-19 for harms analyses. We will search websites of regulatory agencies, trial registries, bibliographic databases, preprint servers and contact trial sponsors to obtain all available data, including unpublished clinical data, for all eligible trials. Our primary outcomes will be all-cause mortality and serious adverse events. Our secondary outcomes will be length of hospital stay, time to death, severe disease, and adverse events. We will assess the risk of bias using the Cochranes Risk of Bias 2 tool and the risk of bias due to missing evidence (e.g. publication bias, selective reporting bias) using the ROB-ME tool. Where appropriate we will synthesise study results by conducting random-effects meta-analysis. We will present our findings in a Summary of Findings table and rate the certainty of the evidence using the GRADE approach. DISCUSSION: By conducting a comprehensive systematic review including unpublished data (where available), we expect to be able to provide valuable information for patients and clinicians about the benefits and harms of remdesivir for the treatment of Covid-19. This will help to ensure optimal treatment for individual patients and optimal utilisation of health care resources. SYSTEMATIC REVIEW REGISTRATION: CRD42021255915.

Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation.

OBJECTIVE: To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications. DESIGN: Cross-sectional study. SETTING: International medical literature. PARTICIPANTS: Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020. MAIN OUTCOME MEASURES: Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably). RESULTS: Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study. CONCLUSIONS: The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.